New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issue 2563: Fix SST file corruption. #2564
Conversation
The record is sent asynchronously. We need to use a copy of the passed buffer in the record. The ownership is retained by the caller and will be potentially changed by the caller. In case of corruption the later blocks were overwritting the previous blocks resulting in corruption
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense. Thank you!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lgtm
@dlg99 we are using the dev/bk-merge-pr.py script in order to merge and cherry pick. Feel free to commit as soon as CI passes. We should cherry pick to branch-4.12 branch-4.11 and branch-4.10 |
Fix SST File corruption during checkpointing ### Motivation Since the SST files are shared among checkpoints, this will not be resolved by future checkpoints. We will fail to restore all future checkpoints that depend on this file. ### Changes The record is sent asynchronously. We need to use a copy of the passed buffer in the record. The ownership is retained by the caller and will be potentially changed by the caller. In case of corruption the later blocks were overwriting the previous blocks resulting in corruption Master Issue: #2563 Reviewers: Andrey Yegorov <None>, Enrico Olivelli <eolivelli@gmail.com>, Matteo Merli <mmerli@apache.org> This closes #2564 from sursingh/fix-sst-corruption, closes #2563
Fix SST File corruption during checkpointing ### Motivation Since the SST files are shared among checkpoints, this will not be resolved by future checkpoints. We will fail to restore all future checkpoints that depend on this file. ### Changes The record is sent asynchronously. We need to use a copy of the passed buffer in the record. The ownership is retained by the caller and will be potentially changed by the caller. In case of corruption the later blocks were overwriting the previous blocks resulting in corruption Master Issue: #2563 Reviewers: Andrey Yegorov <None>, Enrico Olivelli <eolivelli@gmail.com>, Matteo Merli <mmerli@apache.org> This closes #2564 from sursingh/fix-sst-corruption, closes #2563
Fix SST File corruption during checkpointing
Motivation
Since the SST files are shared among checkpoints, this will not be resolved by future checkpoints. We will fail to restore all future checkpoints that depend on this file.
Changes
The record is sent asynchronously. We need to use a copy of the passed buffer
in the record. The ownership is retained by the caller and will be potentially
changed by the caller. In case of corruption the later blocks were
overwriting the previous blocks resulting in corruption
Master Issue: #2563